The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields

reflections, shadows, or fragmented occlusion (for example, looking through the branches of a tree). The assumpMost approaches for estimating optical flow assume that, within a finite image region, only a single motion is present. tion may also be violated when the region spans a depth This single motion assumption is violated in common situations discontinuity in the scene. The standard spatial coherence involving transparency, depth discontinuities, independently constraint suffers from the same problem. It assumes that moving objects, shadows, and specular reflections. To robustly the flow within a neighborhood changes gradually since it estimate optical flow, the single motion assumption must be is caused by a single motion. Optical flow, however, is relaxed. This paper presents a framework based on robust estitypically only piecewise smooth since depth boundaries in mation that addresses violations of the brightness constancy the scene may give rise to discontinuities in the flow field and spatial smoothness assumptions caused by multiple moas a result of motion parallax or the independent motion tions. We show how the robust estimation framework can be of objects. In these common situations, most current apapplied to standard formulations of the optical flow problem proaches cannot recover the multiple motions, give erronethus reducing their sensitivity to violations of their underlying ous motion estimates for the region, and fail to signal that assumptions. The approach has been applied to three standard the underlying assumption of a single motion has been viotechniques for recovering optical flow: area-based regression, lated. correlation, and regularization with motion discontinuities. The fundamental problem is that the estimation of optiThis paper focuses on the recovery of multiple parametric motion models within a region, as well as the recovery of piecewisecal flow involves the pooling of constraints over some spasmooth flow fields, and provides examples with natural and tial neighborhood. To gain accurate estimates this region synthetic image sequences.  1996 Academic Press, Inc. must be sufficiently large to constrain the solution (the standard aperture problem), yet the larger the region of integration the more likely it is to contain multiple motions